United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


10/684,030 


10/09/2003 


Michael E. Goss 


200315391-1 


3183 



7590 06/04/2007 

HEWLETT-PACKARD COMPANY 
Intellectual Property Administration 
P.O. Box 272400 
Fort Collins, CO 80527-2400 



EXAMINER 



PAPPAS, PETER 



ART UNIT 



PAPER NUMBER 



2628 



MAIL DATE 



DELIVERY MODE 



06/04/2007 



PAPER 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



Office Action Summary 


Application No. 

10/684,030 


Applicant(s) 

GOSS ET AL. 


Examiner 

Peter-Anthony Pappas 


Art Unit 

2628 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )[3 Responsive to communication(s) filed on 09 October 2003 . 
2a)D This action is FINAL. 2b)E3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) S Claim(s) 1-33 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) Q Claim(s) is/are allowed. 

6) S Claim(s) 1-33 is/are rejected. 

7) Q Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 29 December 2003 is/are: a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) [3 Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5 > □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 08-06) 



Office Action Summary 



Part of Paper No./Mail Date 20070516 



Application/Control Number: 1 0/684,030 Page 2 
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DETAILED ACTION 
Claim Rejections - 35 USC §112 

1 . The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

2. Claims 2, 3, 6, 14, 22, 23 and 26 are rejected under 35 U.S.C. 112, second 
paragraph, as being indefinite forfaiting to particularly point out and distinctly claim the 
subject matter which applicant regards as the invention. 

3. Claim 2 recites the limitation "...wherein said providing access to a source of said 
visual data..." (lines 1-2). There is insufficient antecedent basis for this limitation in the 
claim. 

4. Claim 3 recites the limitation "...wherein said generating visual data streams..." 
(line 1). There is insufficient antecedent basis for this limitation in the claim. 

5. Claim 6 recites the limitation "...wherein said geometric object..." (line 1). There 
is insufficient antecedent basis for this limitation in the claim. 

6. Claim 14 recites the limitation "...wherein said source..." (line 1). There is 
insufficient antecedent basis for this limitation in the claim. 

7. Claim 22 recites the limitation "...wherein said providing access to a source of 
said visual data..." (lines 1-2). There is insufficient antecedent basis for this limitation in 
the claim. 

8. Claim 23 recites the limitation "...wherein said generating visual data streams..." 
(line 1). There is insufficient antecedent basis for this limitation in the claim. 
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9. Claim 26 recites the limitation "...wherein said geometric object..." (line 1). 
There is insufficient antecedent basis for this limitation in the claim. 

Claim Rejections - 35 USC § 102 

10. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

11. Claims 1-4, 7-15, 17-24 and 27-33 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Jain et al. (U.S. Patent No. 5, 745, 126). 

12. In regard to claim 1 Jain et al. teaches a method and system in which a viewer 
can command the selection of real, or even the synthesis of virtual, video images of the 
scene in response to any of his or her desired and selected (i) spatial perspective on the 
scene, (ii) static or dynamically moving object appearing in the scene, or (iii) event 
depicted in the scene, (col. 7, lines 37-42; col. 8, lines 41-51). Jain et al. teaches 
synthesizing a virtual video camera, and a virtual video image, from multiple real video 
images obtained by multiple real video cameras (col. 1 , lines 24-27). 

Jain et al. teaches that a computer receives from a prospective user/viewer of the 
scene a user/viewer-specified criterion relative to which criterion the user/viewer wishes 
to view the scene. From a (i) 3D model and (ii) the criterion, the computer produces a 
particular 2D image of the scene that is in accordance with the user/viewer-specified 
criterion. This particular 2D image of the real-world scene is then displayed on a video 
display to the user/viewer (col. 9, lines 8-16). However, the computer is not limited to 
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selecting from the 3D model a 2D image that is, or that corresponds to, any of the 
images of the real-world scene as was imaged by any of the multiple video cameras. 
Instead, the computer may synthesize from the 3D a completely new 2D image that is 
without exact equivalence to any of the images of the real-world scene as have been 
imaged by any of the multiple video cameras (col. 9, lines 57-61). It is noted said 2D 
images is considered to comprise a view volume with a defined field of view. 

Jain et al. illustrates in Fig. 13 a graphical illustration showing the intersection 
formed by the rectangular viewing frustum of each camera scene onto the environment 
volume. The filled frustum representing possible areas where the object can be located 
in the 3D model while, by use of multiple views, the intersection of the frustum from 
each camera will closely approximate the 3D location and form of the object in the 
environment model (col. 11, lines 66-67; col. 12, lines 1-7). 

Jain et al. teaches that a viewer interface permits said viewer to select the 
perspective that he or she wants (col. 18, lines 7-8), wherein said interface includes a 
3D cursor that moves in 3D space (col. 22, lines 31-35). It is noted that said cursor is 
considered to represent a participant (e.g. user/viewer) within said virtual environment. 

Jain et al. teaches that the footprint of a bounding box is projected to the primary 
surface of motion by intersecting a ray drawn from the optic center of that particular 
camera through the foot of the bounding box with the ground surface. All supporting 
observations are used (with appropriate weighting based on distance from the camera, 
direction of motion, etc.) to update the position of each object (col. 33, lines 53-65). 
Jain et al. implicitly teaches determining a proximity of a representation of an observed 
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object in said virtual environment to said view volume. For example, as disclosed 
above Jain et al. teaches a plurality of viewing frustums specific to respective cameras 
from which said viewing volume is generated. Whether or not said viewing volume is a 
view volume derived from one of said cameras or generated synthetically from a 
plurality of said cameras said volume has a bounds and the objects contained within 
said bounds must be determined to be rendered. It is noted that the respective claim 
language is silent as to the value. or range of values that define said proximity. 

Jain et al. implicitly teaches processing a view dependent visual data stream of 
said observed object only when said representation is within a specified proximity to 
said view volume. For example, an observed object (e.g. a tracked football player - 
col. 7, lines 51-64) located within a given displayed view volume (e.g. view volume of a 
camera - col. 7, lines 65-67; col. 8, lines 1-9) would only be displayed while said object 
is contained in said view volume. 

13. In regard to claim 2 Jain et al. teaches synthesizing a virtual video camera, and a 
virtual video image, from multiple real video images obtained by multiple (viewpoints) 
real video cameras (col. 1 , lines 24-27). Objects of interest in the scene are identified 
and classified in these 2D images. These multiple 2D images of the scene, and their 
accompanying object information, are then combined in a computer into a 3D video 
database, or model, of the scene (col. 8, lines 55-60). The ultimate response of the MPI 
video system is to synthesize the exact synthetic image, and image sequence, the 
viewer desires and demands. Even here, no image can be formed where no source 
image data exists (col. 19, lines 12-16). 
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14. In regard to claim 3 the rationale disclosed in the rejection of claims 1 and 2 is 
incorporated herein. It is noted that a new view synthesis technique is considered to 
read on a new viewpoint (e.g. the selection of a new viewpoint by said user). 

15. In regard to claim 4 it is noted said display information is sent to said user for 
display (Fig. 1, element 18). 

16. In regard to claim 7 Jain et al. teaches that an object that is out of view, too 
small, and/or occluded from view in one camera is in view, large and/or un-occluded to 
the view of another camera (col. 34, lines 43-45). Jain et al further teaches that camera 
handoff should be understood to be the event in which a dynamic object passes from 
one camera coverage zone to another. The multi-perspective perception system must 
maintain a consistent representation of an object's identity and behavior during camera 
handoff (col. 29, lines 59-63). It is noted that when said camera handoff occurs that the 
display from the camera from which focus has been passed is ceased to be displayed. 

17. In regard to claim 8 Jain et al. teaches maintaining video information even though 
it is no longer associated with a given displayed object - e.g. hysteresis (col. 8, lines 55- 
60; col. 14, lines 60-64; col. 18, lines 27-30). It is implicitly taught that Jain et al. is able 
to determine in advance if video information is to be needed - e.g. anticipation (col. 25, 
lines 5-11). It is noted that the area surrounding, but not necessarily displayed (e.g. the 
entire football field), for said object (e.g. a tracked football player) is considered to read 
on an extended boundary. 

18. In regard to claim 9 the rationale disclosed in the rejection of claim 3 is 
incorporated herein. It is noted that when a perspective is selected by a user, be said 
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selected perspective the first perspective chosen or a different perspective then the 
current one, that a view volume is generated for said respective perspective. 

19. In regard to claim 10 the rationale disclosed in the rejection of claim 7 is 
incorporated herein. 

20. In regard to claim 11 it is noted a tracked football player is considered a local 
participant (e.g. said football player is localized to the respective football game said 
player is participating in). 

21. In regard to claim 12 Jain et al. teaches a virtual collaborative environment that 
be utilized by a plurality (N-way) of users (col. 14, lines 43-59). 

22. In regard to claim 13 the rationale disclosed in the rejection of claim 1 is 
incorporated herein. 

23. In regard to claim 14 the rationale disclosed in the rejection of claim 3 is 
incorporated herein. 

24. In regard to claim 15 the rationale disclosed in the rejection of claim 4 is 
incorporated herein. 

25. In regard to claim 17 the rationale disclosed in the rejection of claim 7 is 
incorporated herein. 

26. In regard to claim 18 the rationale disclosed in the rejection of claim 8 is 
incorporated herein. 

27. In regard to claim 19 the rationale disclosed in the rejection of claim 10 is 
incorporated herein. It is noted the resulting display from said change of camera 



Application/Control Number: 10/684,030 Page 8 

Art Unit: 2628 

(change of location of said user/viewer) is considered to read on a change of view 
volume. 

28. In regard to claim 20 the rationale disclosed in the rejection of claim 10 is 
incorporated herein (e.g. said tracked football player). 

29. In regard to claim 21 Jain et al. teaches that a configuration of the MPI football 
video/television system is shown in Fig. 3. The current system consists of a UNIX 
workstation, a laser disc player, a video capture board, and a TV monitor and graphical 
display (col. 21, lines 37-42). Jain et al. teaches that said system utilizes 
microprocessors (col. 18, lines 52-56) and that said method is implemented by a 
programmed computer processes (col. 17, lines 51-67; col. 18, lines 1-44). It is 
implicitly taught that said processes, to be executed by a processor(s), are stored on 
computer readable medium (e.g. memory) connected to said processors as said 
processes must be stored in some form computer readable medium as well as able to 
be delivered to said processors to be processed. 

30. In regard to claim 22 the rationale disclosed in the rejection of claim 2 is 
incorporated herein. 

31 . In regard to claim 23 the rationale disclosed in the rejection of claim 3 is 
incorporated herein. 

32. In regard to claim 24 the rationale disclosed in the rejection of claim 4 is 
incorporated herein. 

33. In regard to claim 27 the rationale disclosed in the rejection of claim 7 is 
incorporated herein. 
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34. In regard to claim 28 the rationale disclosed in the rejection of claim 8 is 
incorporated herein. 

35. In regard to claim 29 the rationale disclosed in the rejection of claim 9 is 
incorporated herein. 

36. In regard to claim 30 the rationale disclosed in the rejection of claim 10 is 
incorporated herein. 

37. In regard to claim 31 the rationale disclosed in the rejection of claim 1 1 is 
incorporated herein. 

38. In regard to claim 32 the rationale disclosed in the rejection of claim 12 is 
incorporated herein. 

39. In regard to claim 33 the rationale disclosed in the rejection of claim 1 is 
incorporated herein. Jain et al. teaches that said method is implemented by a 
programmed computer processes (col. 17, lines 51-67; col. 18, lines 1-44). It is noted 
said processes are considered to read on instructions. It is implicitly taught that said 
processes, to be executed by a processor(s), are stored on computer readable medium 
(e.g. memory) as said processes must be stored in some form computer readable 
medium to be processed. 

Claim Rejections - 35 USC § 103 

40. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said. subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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41. Claims 5, 6, 16, 25 and 26 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Jain et al. (U.S. Patent No. 5, 745, 126), as applied to claims 1-4, 7- 
15, 17-24 and 27-33, in view of Sullivan (U.S. Patent No. 6, 100, 862). 

42. In regard to claim 5 Jain et al. fails to explicitly teach wherein said view volume 
comprises a series of expanding cross-sections of a geometric object. Sullivan teaches 
a multi-planar volumetric display (MVD) system and method of operation are disclosed 
which generate volumetric 3D images (col. 2, lines 56-58). Sullivan teaches wherein a 
view volume comprises a series of expanding cross-sections of a geometric object (col. 
6, lines 22-33; Fig. 1). It is noted that, at least in part, said user-selected perspective is 
considered to determined a view direction. 

It would have been obvious to one skilled in the art, at the time of the Applicant's 
invention, to incorporate the teachings of Sullivan into the system taught by Jain et al., 
because through such incorporation it would provide an improved means of presenting 
the display information generated by Jain et al. as said information would be presented 
in a more realistic, e.g. 3D, manner. 

43. In regard to claim 6 Jain et al. teaches that the system receives and processes 
multiple video views (images) generally of the football field (col. 7, lines 55-57). It is 
noted a football field is considered a four-sided rectangular plane. 

44. In regard to claim 16 the rationale disclosed in the rejection of claim 5 is 
incorporated herein. 

45. In regard to claim 25 the rationale disclosed in the rejection of claim 5 is 
incorporated herein. 
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46. In regard to claim 26 the rationale disclosed in the rejection of claim 6 is 
incorporated herein. 

Conclusion 

47. The prior art made of record and not relied upon is considered pertinent to 
Applicant's disclose: Foley et al. (specifically - p. 237-242; Fig. 6.46). 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Peter-Anthony Pappas whose telephone number is 571- 
272-7646. The examiner can normally be reached on M-F 9:00am-5:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka Chauhan can be reached on 571-272-7782. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Peter-Anthony Pappas 

Examiner 

Art Unit 2628 
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